LocVTP: Video-Text Pre-training for Temporal Localization

Video-Text Pre-training (VTP) aims to learn transferable representations for various downstream tasks from large-scale web videos. To date, almost all existing VTP methods are limited retrieval-based tasks, e.g., video retrieval, whereas their transfer potentials on localization-based temporal grounding, under-explored. In this paper, we experimentally analyze and demonstrate the incompatibility of current with localization propose a novel Localization-oriented framework, dubbed as LocVTP. Specifically, perform fine-grained contrastive alignment complement coarse-grained one by clip-word correspondence discovery scheme. further enhance reasoning ability learned feature, context projection head aware loss perceive contextual relationships. Extensive experiments four across six datasets that our LocVTP achieves state-of-the-art performance both tasks. Furthermore, conduct comprehensive ablation studies thorough analyses explore optimum model designs training strategies. Codes available at https://github.com/mengcaopku/LocVTP .